Systems and methods for analyzing facial expressions within an online classroom to gauge participant attentiveness

ABSTRACT

Systems, methods, and non-transitory computer readable analyzing facial expressions within an interactive online event to gauge participant level of attentiveness are provided. Facial expressions from a plurality of participants accessing an interactive online event may be analyzed to determine each participant&#39;s facial expression. The determined expressions may be analyzed to determine an overall level of attentiveness. The level of attentiveness be relayed to the host of the interactive online event to inform him or her how the participants are reacting to the interactive online event. If there are participants not paying attention or confused, the host may modify their presentation to increase the attentiveness of the students.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.62/106,842, filed on Jan. 23, 2015, the disclosure of which isincorporated herein by reference in its entirety.

FIELD OF THE INVENTION

This disclosure generally relates to systems and methods for analyzingfacial expressions within an online classroom to gauge participantattentiveness, interest, and/or comprehension.

BACKGROUND OF THE INVENTION

Massive Open Online Courses (“MOOCs”) are quickly becoming a majorfactor in the world of education. The ability to log into a class fromone's own personal computer or mobile device enables individuals toparticipate in the learning process regardless of their location. Theexpansion of MOOCs has also been aided, in large part, to theadvancements in mobile devices, such as laptop computers, tablets, andsmartphones, which are rapidly becoming more and more powerful androbust with each passing day.

However, a significant drawback to large MOOCs, or other largeinteractive online events, is the difficulty a teacher or presenter mayencounter trying to effectively transmit the information being presentedto the students/participants. In a small classroom, for example, ateacher is capable of seeing each student's face/body to gauge thatstudent's or students' attentiveness. A confused facial expression onmultiple students could mean that the subject matter, topic, or deliverytechnique, may not be working effectively and the teacher may augment ormodify their approach accordingly. In a large online classroom, however,the ability to even see each student's face may not be possible (it maynot be possible even in a small online “classroom” as well).

Thus, it would be beneficial for there to be systems and methods thatallow a teacher or presenter for a massive online event, such as a MOOC,to accurately gauge a level of attentiveness, interest, and/orcomprehension of the participants to aid in effectively delivering theintended message.

SUMMARY OF THE INVENTION

Systems and methods for analyzing facial expression within an onlineclassroom to gauge participant attentiveness, interest, and/orcomprehension are described herein.

In one embodiment, a method for monitoring participants' level ofattentiveness within an online event is provided. In the exemplaryembodiment, a plurality of videos from a plurality of participantsaccessing an interactive online event may be received. In thisembodiment, each received video may correspond to a differentparticipant. At least one facial image from each video may be captured.Each captured facial image may then be analyzed by comparing each facialimage to a plurality of predefined facial expressions. Each facial imagemay then be matched to at least one predefined facial expression. Eachfacial image can then be assigned a value that represents the matchedpredefined facial expression. Next, a level of attentiveness of theinteractive online event may be determined by processing each of theassigned values together. The level of attentiveness may then beprovided to a host device accessing the interactive online event.

In another embodiment, a method for monitoring a participant's level ofattentiveness within an online event is provided. In the exemplaryembodiment, a video from a participant accessing an interactive onlineevent is received. A facial image from the received video is captured.The captured facial image may then be analyzed by comparing the facialimage to a plurality of predetermined facial expressions and matchingthe facial image to at least one predetermined facial expression. Thefacial image may then be assigned a value that represents the matchedfacial expressions. Next, a level of attentiveness of the interactiveonline event is determined by processing the assigned values together.The level of attentiveness is then provided to a host device accessingthe interactive online event.

In another embodiment, a system for monitoring participants' level ofattentiveness within an interactive online event is provided. Thesystem, in this embodiment, may include a plurality of user devicesaccessing an interactive online event. Each user device may correspondto a participant accessing the interactive online event. The system mayalso include a host device accessing the interactive online event. Thehost device may correspond to a host of the interactive online event.The system may also include a server. The server may be operable toreceive a plurality of videos from the user devices. The server may alsobe operable to capture at least one facial image from each receivedvideo. The server may be further operable to analyze the captured facialimages. Each captured facial image may then be analyzed by comparingeach facial image to a plurality of predefined facial expressions. Eachfacial image may then be matched to at least one predefined facialexpression. Each facial image can then be assigned a value thatrepresents the matched facial expression. The server may be furtheroperable to determine a level of attentiveness for the interactiveonline event by processing each assigned value. The server may befurther operable to transmit the determined level of attentiveness tothe host device.

BRIEF DESCRIPTION OF THE DRAWINGS

It is noted that the U.S. patent or application file contains at leastone drawing executed in color. Copies of this patent or patentapplication publication with color drawings will be provided by the U.S.Patent Office upon request and payment of the necessary fee. The aboveand other features of the present invention, its nature and variousadvantages will be more apparent upon consideration of the followingdetailed description, taken in conjunction with the accompanyingdrawings in which:

FIG. 1 is an illustrative diagram of a system in accordance with variousembodiments;

FIG. 2 is an illustrative block diagram of an exemplary device inaccordance with various embodiments;

FIG. 3 is an illustrative diagram of an exemplary user interfacepresenting a plurality of participants from an online event inaccordance with various embodiments;

FIG. 4 is an illustrative diagram of an exemplary user interfacepresenting a plurality of participants from an online event inaccordance with various embodiments;

FIG. 5 is an illustrative diagram of an exemplary user interfacepresenting a plurality of participants from an online event inaccordance with various embodiments;

FIG. 6 is an illustrative diagram of an exemplary user interfacepresenting a plurality of participants from an online event inaccordance with various embodiments;

FIG. 7 is an illustrative flowchart of an exemplary process inaccordance with various embodiments; and

FIG. 8 is an illustrative flowchart of an exemplary process inaccordance with various embodiments.

DETAILED DESCRIPTION OF THE INVENTION

The present invention may take form in various components andarrangements of components, and in various techniques, methods, orprocedures and arrangements of steps. The referenced drawings are onlyfor the purpose of illustrated embodiments, and are not to be construedas limiting the present invention. Various inventive features aredescribed below that can each be used independently of one another or incombination with other features.

It should be noted that, as used herein, the terms “participantattentiveness” or “student attentiveness” are not to be construed aslimited, and may correspond to any characterization or method foraccessing an individual's interest in, ability to view, and/orcapability to retain information being presented to them. Therefore, itshould also be noted that, as used herein, the terms “class”,“classroom”, and/or “event” are not limited to be related to aneducational process. For example, such applicable situations where anindividual's attentiveness is analyzed may correspond to educationalpurposes, business purposes, legal purposes, entertainment purposes,marketing purposes, scientific purposes, or any other type of situation,or any combination thereof.

FIG. 1 is an illustrative diagram of a system in accordance with variousembodiments. System 100 may include server 102, user devices 104, andhost device 108, which may communicate with one another across network106. Although only three user devices 104, one host device 108, and oneserver 102 are shown within FIG. 1, persons of ordinary skill in the artwill recognize that any number of user devices, host devices, and/orservers may be used. Furthermore, in some embodiments, one or more ofuser devices 104, host device 108, and server 102 may not be included.For example, system 100 may include user devices 104 and server 102, andno host device 108 may be present.

Server 102 may correspond to one or more servers capable of facilitatingcommunications and/or servicing requests from user devices 104 and/orhost device 108. User device 104 may send and/or receive data fromserver 102 and/or host device 104 via network 108. Similarly, hostdevice 108 may send and/or receive data from server 102 and/or userdevices 104 via network 106. In some embodiments, network 106 mayfacilitate communications between one or more user devices 104.

Network 106 may correspond to any network, combination of networks, ornetwork devices that may carry data communications. For example, network106 may be any one or combination of local area networks (“LAN”), widearea networks (“WAN”), telephone networks, wireless networks,point-to-point networks, star networks, token ring networks, hubnetworks, ad-hoc multi-hop networks, or any other type of network, orany combination thereof. Network 106 may support any number of protocolssuch as WiFi (e.g., 802.11 protocol), Bluetooth®, radio frequencysystems (e.g., 900 MHZ, 1.4 GHZ, and 5.6 GHZ communication systems),cellular networks (e.g., GSM, AMPS, GPRS, CDMA, EV-DO, EDGE, 3GSM, DECT,IS-136/TDMA, iDen, LTE, or any other suitable cellular networkprotocol), infrared, TCP/IP (e.g., any of the protocols used in each ofthe TCP/IP layers), HTTP, BitTorrent, FTP, RTP, RTSP, SSH, Voice over IP(“VOIP”), or any other communication protocol, or any combinationthereof. In some embodiments, network 106 may provide wiredcommunications paths for user devices 104 and/or host device 108.

User device 104 may correspond to any electronic device or systemcapable of communicating over network 106 with server 102, host device108, and/or with one or more additional user devices 104. For example,user devices 104 may be a portable media players, cellular telephones,pocket-sized personal computers, personal digital assistants (“PDAs”),desktop computers, laptop computers, wearable electronic devices,accessory devices, and/or tablet computers. User devices 104 may includeone or more processors, storage, memory, communications circuitry,input/output interfaces, as well as any other suitable component, such afacial recognition module. Furthermore, one or more components of userdevices 104 may be combined or omitted.

Host device 108 may correspond to any electronic device or systemcapable of communicating over network 106 with server 102 or user device104. For example, host device 108 may a portable media players, cellulartelephones, pocket-sized personal computers, personal digital assistants(“PDAs”), desktop computers, laptop computers, and/or tablet computers.Host device 108 may include one or more processors, storage, memory,communications circuitry, input/output interfaces, as well as any othersuitable component, such a facial recognition module. Furthermore, oneor more components of host device 108 may be combined or omitted.

Although examples of embodiments may be described for a user-servermodel with a server servicing requests of one or more user applications,persons of ordinary skill in the art will recognize that any other model(e.g., peer-to-peer) may be available for implementation of thedescribed embodiments. For example, a user application executed on userdevice 104 may handle requests independently and/or in conjunction withserver 102.

FIG. 2 is an illustrative block diagram of an exemplary device inaccordance with various embodiments. Device 200 may, in someembodiments, correspond to one of user devices 104 and/or host device108 of FIG. 1. It should be understood by persons of ordinary skill inthe art, however, that device 200 is merely one example of a device thatmay be implemented within a server-device system, and it is not limitedto being only one part of the system. Furthermore, one or morecomponents included within device 200 may be added or omitted.

In some embodiments, device 200 may include processor 202, storage 204,memory 206, communications circuitry 208, input interface 210, outputinterface 216, and facial recognition module 222. Input interface 210may, in some embodiments, include camera 212 and microphone 214. Outputinterface 216 may, in some embodiments, include display 218 and speaker220. In some embodiments, one or more of the previously mentionedcomponents may be combined or omitted, and/or one or more components maybe added. For example, memory 204 and storage 206 may be combined into asingle element for storing data. As another example, device 200 mayadditionally include a power supply, a bus connector, or any otheradditional component. In some embodiments, device 200 may includemultiple instances of one or more of the components included therein.However, for the sake of simplicity, only one of each component has beenshown within FIG. 2.

Processor 202 may include any processing circuitry, such as one or moreprocessors capable of controlling the operations and functionality ofdevice 200. In some embodiments, processor 202 may facilitatecommunications between various components within device 200. Processor202 may run the device's operating system, applications resident on thedevice, firmware applications, media applications, and/or any other typeof application, or any combination thereof. In some embodiments,processor 202 may process one or more inputs detected by device 200 andperform one or more actions in response to the detected inputs.

Storage 204 may correspond to one or more storage mediums. Various typesof storage mediums may include, but are not limited to, hard-drives,solid state drives, flash memory, permanent memory (e.g., ROM), or anyother storage type, or any combination thereof. Any form of data orcontent may be stored within storage 204, such as photographs, musicfiles, videos, contact information, applications, documents, or anyother file, or any combination thereof. Memory 206 may include cachememory, semi-permanent memory (e.g., RAM), or any other memory type, orany combination thereof. In some embodiments, memory 206 may be used inplace of and/or in addition to external storage for storing data ondevice 200.

Communications circuitry 208 may include any circuitry capable ofconnecting to a communications network (e.g., network 106) and/ortransmitting communications (voice or data) to one or more devices(e.g., user devices 104 and/or host device 108) and/or servers (e.g.,server 102). Communications circuitry 208 may interface with thecommunications network using any suitable communications protocolincluding, but not limited to, Wi-Fi (e.g., 802.11 protocol),Bluetooth®, radio frequency systems (e.g., 900 MHz, 1.4 GHz, and 5.6 GHzcommunications systems), infrared, GSM, GSM plus EDGE, CDMA, quadband,VOIP, or any other protocol, or any combination thereof.

Input interface 210 may include any suitable mechanism or component forreceiving inputs from a user operating device 200. Input interface 210may also include, but is not limited to, an external keyboard, mouse,joystick, musical interface (e.g., musical keyboard), or any othersuitable input mechanism, or any combination thereof.

In some embodiments, user interface 210 may include camera 212. Camera212 may correspond to any image capturing component capable of capturingimages and/or videos. For example, camera 212 may capture photographs,sequences of photographs, rapid shots, videos, or any other type ofimage, or any combination thereof. In some embodiments, device 200 mayinclude one or more instances of camera 212. For example, device 200 mayinclude a front-facing camera and a rear-facing camera. Although onlyone camera is shown in FIG. 2 to be within device 200, it persons ofordinary skill in the art will recognize that any number of cameras, andany camera type may be included. Additionally, persons of ordinary skillin the art will recognize that any device that can capture images and/orvideo may be used. Furthermore, in some embodiments, camera 212 may belocated external to device 200.

In some embodiments, device 200 may include microphone 214. Microphone214 may be any component capable of detecting audio signals. Forexample, microphone 214 may include one more sensors or transducers forgenerating electrical signals and circuitry capable of processing thegenerated electrical signals. In some embodiments, user device mayinclude one or more instances of microphone 214 such as a firstmicrophone and a second microphone. In some embodiments, device 200 mayinclude multiple microphones capable of detecting various frequencylevels (e.g., high-frequency microphone, low-frequency microphone,etc.). In some embodiments, device 200 may include one or externalmicrophones connected thereto and used in conjunction with, or insteadof, microphone 214.

Output interface 216 may include any suitable mechanism or component forgenerating outputs from a user operating device 200. In someembodiments, output interface 216 may include display 218. Display 218may correspond to any type of display capable of presenting content to auser and/or on a device. Display 218 may be any size and may be locatedon one or more regions/sides of device 200. For example, display 218 mayfully occupy a first side of device 200, or may occupy a portion of thefirst side. Various display types may include, but are not limited to,liquid crystal displays (“LCD”), monochrome displays, color graphicsadapter (“CGA”) displays, enhanced graphics adapter (“EGA”) displays,variable graphics array (“VGA”) displays, or any other display type, orany combination thereof. In some embodiments, display 218 may be a touchscreen and/or an interactive display. In some embodiments, the touchscreen may include a multi-touch panel coupled to processor 202. In someembodiments, display 218 may be a touch screen and may includecapacitive sensing panels. In some embodiments, display 218 may alsocorrespond to a component of input interface 210, as it may recognizetouch inputs.

In some embodiments, output interface 216 may include speaker 220.Speaker 220 may correspond to any suitable mechanism for outputtingaudio signals. For example, speaker 220 may include one or more speakerunits, transducers, or array of speakers and/or transducers capable ofbroadcasting audio signals and audio content to a room where device 200may be located. In some embodiments, speaker 220 may correspond toheadphones or ear buds capable of broadcasting audio directly to a user.

Device 200 may, in some embodiments, also include facial recognitionmodule 222. Facial recognition module 222 may allow device 200 toanalyze a user or users facial expression(s), and determine a type ofmood or level of attentiveness associated with the user or users basedon the analysis. Facial recognition module 222 may receive an image fromcaptured from camera 212 or display 218 (e.g., video chat windows withina user interface presented on display 218), and determine a facialexpression for the user or users depending on various facialcharacteristics determined therefrom.

For example, facial recognition module 222 may be able to determine thatthe image or images include a user who is smiling, nodding, furrowingtheir brow, crying, or displaying any other type of emotion. The variousfacial expressions determined to be present within the image or imagesmay then be stored in facial expression database 224, for example.Facial expression database 224 may also, in some embodiments, be used tocompare captured images with pre-defined facial expressions to determinewhat a particular user or users facial expression is. For example, areceived image of a user may be compared against pre-defined images of auser smiling, nodding, furrowing their brow, crying, etc., to determinewhether the user within the received image is expressing any one ofthese emotions. In some embodiments, facial expression database 224 mayalso learn or receive new expressions, or update how a particularexpression is characterized based on the received images.

In some embodiments, facial recognition module 222 may also includeanalysis software executable on device 200 that allows a user to takethe determined facial expressions and generate data signifying anemotion or level of attentive of the individuals within the images. Forexample, a teacher viewing video chat windows corresponding to studentswithin an online classroom may be able to determine, based on the facialexpressions of each student seen and captured within each students'video chat window, a level of attentiveness for the students. In thisway, a teacher may be better able to gauge whether the students areunderstanding the material being presented in real-time, andappropriately modify or augment their presentation to enhance thelearning experience.

FIG. 3 is an illustrative diagram of an exemplary user interface inaccordance with various embodiments. User interface 300 may, in someembodiments, correspond to a user interface displayed on host device108. For example, host device 108 may be a device operated by a teacherof an online classroom, and video chat windows 301-309 may correspond tovideo chat windows for students participating in the online class.Although only nine video chat windows are displayed within userinterface 300, any number of video chat windows may be included (e.g.,50, 100, 1,000, etc.).

Each video chat window 301-309 may include an image of that particularuser's face and/or body, as well as a user identifier for thecorresponding user. For example, video chat window 304 includes facialimage 314, corresponding to user identifier 324—“USER 4”. In someembodiments, the device displaying user interface 300 (e.g., host device108), may analyze each facial image 314 corresponding to each video chatwindow displayed therein to determine a facial expression of that videochat window's corresponding user. However, in some embodiments, thedevice displaying user interface 300 and/or the server may analyze eachuser accessing an online event, such as the online class previouslymentioned, to determine each user's facial expression. If the number ofparticipants in the online event is very large, it may be difficult toview them all at a same time on one user interface, therefore, eachvideo feed from each user accessing the online event may be analyzed insome embodiments, regardless of whether or not the corresponding videofeed is displayed within a video chat window in user interface 300.However, for simplicity, the facial expressions of users correspondingto displayed video chat windows will be described.

As shown in user interface 300, each facial image of each video chatwindow 301-309 corresponds to a facial expression that may becharacterized as smiling or happy, for example. This information may bedetermined using facial recognition module 222 of FIG. 2, and may thenbe converted to numerical data or some other form of indicator (e.g.,visual, audio), such as one of a variety of colors, for the host orpresenter of the online event. For example, if all of the usersaccessing the online event corresponding to user interface 300 aresmiling, facial recognition module 222 will display data to thepresenter signifying that a large percentage (e.g., 90% or greater) arefavorably receiving the information being presented to them. Forexample, all of video chat windows 301-309 may be shaded green toindicate to the instructor that the corresponding users appear to beengaged. In some embodiments, user interface 300 may have its background320 change to a color indicating that the majority of students areengaged in the materials being presented. For example, when background320 is green, the instructor will know that the correspondingparticipants are engaged.

In some embodiments, video chat windows 301-309 may display more thanjust a corresponding user's face. For example, a user's face and bodymay also be displayed within their video chat window. In someembodiments, the user's face and body may be analyzed to determine auser's emotion. For example, a user who is slouching may correspond to auser who is not paying attention, whereas a user sitting upright maycorrespond to a user fully engaged in the presentation.

FIG. 4 is an illustrative diagram of an exemplary user interface inaccordance with various embodiments. User interface 400 may, in someembodiments, be substantially similar to user interface 300 of FIG. 3,with the exception that the facial expressions of facial images 414 ofusers displayed within video chat windows 401-409 may be different thanthe facial expression of the users within video chat windows 301309 ofFIG. 3. For example, facial images 414 may correspond to a user, such as“USER 4” 424, frowning or not smiling. In this case, the individual chatwindows or background 420 may turn yellow or red, for example, dependingon the level of dissatisfaction or lack of engagement that facialrecognition module 222 determines the faces of the users displayedwithin video chat windows 401-409 are presenting. For example,background 420 may be red, signifying that the majority of participantsare not engaged. In some embodiments, additional visual or audio cuesmay be provided to the presenter on their host device, where the cuesindicate to the presenter that the users whose images are beingdisplayed within video chat videos 401-409 are not comprehending thepresentation. For example, user interface 400 may flash, beep, blink, orprovide any other audio/visual indicator, or any combination thereof.

Similarly, facial images received from each of video chat windows401-409 may correspond to similar emotional reading of the correspondingusers. Facial recognition module 222 may therefore determine that theusers accessing the online event who displayed by user interface 400 arenot understanding or paying attention to the presented materials, andtherefore data may be sent to the presenter informing them that a lownumber of students (e.g., less than 20%) are understanding the materialbeing discussed. This may allow the presenter to accurately modify theirpresentation so that their content is more favorably received.

FIG. 5 is an illustrative diagram of an exemplary user interface inaccordance with various embodiments. User interface 500, in someembodiments, may be substantially similar to user interfaces 300 and 400of FIGS. 3 and 4, respectively, with the exception that some facialimages of some users may correspond to users having one emotion, whileother users may have a different emotion. For example, user interface500 may include video chat windows 501-509, each including facialimages, such as facial images 511 and 514, and user identifiers, such as“USER 4” 524.

In some embodiments, facial image 511 of video chat window 501 may besubstantially similar to facial image 314 of FIG. 3. Furthermore, videochat windows 503 and 505-508 may also have facial images similar tofacial image 511. This may correspond to each user of the correspondingvideo chat windows having a similar facial expression, which in someembodiments may be smiling. In some embodiments, facial image 514 ofvideo chat window 501 may be substantially similar to facial image 414of FIG. 4. Furthermore, video chat windows 502 and 509 may also havefacial images displayed therein that are similar to facial image 514.This may correspond to each user of the corresponding video chat windowshaving a similar facial expression, which in some embodiments maycorrespond to frowning.

The information corresponding to the various facial expressions of eachuser of video chat windows 501-509 may be analyzed within facialrecognition module 222 of device 200, and converted into data for apresenter or host to use. For example, based on the facial expressionsreceived within video chat windows 501-509 of user interface 500, a hostor presenter may be informed that approximately 33% (e.g., one-third) ofthe participants are disinterested or not paying attention. The host orpresenter may then be able to modify the presented materials to betterallow the entire participation group to comprehend the material. In thiscase, background 520 may, accordingly, turn yellow or some other color,signifying that some participants are paying attention while others arenot. In some embodiments, the color of a partially engaged class mayvary in hue based on the amount of participants that are engaged in theonline event.

FIG. 6 is an illustrative diagram of an exemplary user interface inaccordance with various embodiments. User interface 600 may, in someembodiments, correspond to a user interface displayed on host device108. For example, host device 108 may be a device operated by a teacherof an online classroom, and video chat windows 601-609 may correspond tovideo chat windows for students participating in the online class.Although only nine video chat windows are displayed within userinterface 600, any number of video chat windows may be included (e.g.,50; 100; 1,000; etc.).

Each of video chat windows 601-609 may include an image of thatparticular user's face and/or body (or a portion of that user's faceand/or body), and a user identifier. For example, video chat window 604includes facial image 614, corresponding to user identifier 624—“USER4”. Each video chat window (e.g., chat windows 601-609) also includes afacial image (e.g., facial images 611-619) of the corresponding user,which may be captured (e.g., by camera 212) In some embodiments, thedevice displaying user interface 600 (e.g., host device 108), mayanalyze each facial image (e.g. facial images 611-619) corresponding toeach video chat window (e.g., chat windows 601-609) displayed thereon todetermine a facial expression of that video chat window's correspondinguser. However, in some embodiments, the device displaying user interface600 and/or the server may analyze a facial image of each user accessingan online event, such as the online class previously mentioned, todetermine each user's facial expression. If the number of participantsin the online event is very large, it may be difficult or even notfeasible to view them all of the participants at a same time on a singleuser interface. Therefore, in some embodiments, a video feed from eachaccessing user may be analyzed regardless of whether or not thecorresponding video feed is displayed within a video chat window in userinterface 600. However, for simplicity, only the facial expressions ofusers corresponding to video chat windows 601-609, which are displayedwithin user interface 600, will be described.

As shown in user interface 600, each facial image of each video chatwindow 601-609 includes a facial expression that may be characterized assmiling, or happy. This information may be determined using facialrecognition module 222 of FIG. 2, and may be converted to numerical dataor some other form of indicator (e.g., visual, audio), such as one of avariety of colors, for the host or presenter of the online event. Forexample, because USER 4 in chat window 604 is smiling, indicator 634 maychange to be green. Indicator 634 can change colors as the facialexpression in chat window 604 changes. For example, if USER 4 in chatwindow 604 starts frowning, indicator 634 may turn red. Although onlyone indicator is shown, indicators for each video chat window (e.g.video chat windows 601-609) may be provided. Although indicator 634, inthe illustrative embodiment, is a circle, persons having ordinary skillin the art will recognize that any type of indicator may be used,including but not limited to words, symbols, and letters.

Furthermore, in some embodiments, a message 644 may be displayed withinuser interface 600. Message 644 may indicate an overall status, oremotion, of the individuals accessing the interactive event displayedusing user interface 600. For instance, because video chat windows601-609 show participants that are happy, message 644 may state: “USERSARE HAPPY,” or “ATTENTIVENESS OF USERS IS GOOD.” Message 644 may changeas the facial expressions of one or more of the participants change.Furthermore, in some embodiments, the message may relate to any numberof participants (e.g. “USER 4 IS HAPPY”).

In some embodiments, indicator 634 and message 644 may only be visibleif the host chooses. The interactive online event may give access to theindicators, allowing a host to view them if he or she wishes.Furthermore, indicator 634 and message 644 may be sent or simply storedfor the host to review at a later time.

FIG. 7 is an illustrative flowchart of an exemplary process inaccordance with various embodiments. Process 700 may begin at step 701.At step 701, videos may be received from a plurality of participantsaccessing an interactive online event. For example, as seen in FIG. 6,videos from nine participants are received by a host device (e.g., hostdevice 108), and may be displayed within a user interface (e.g., userinterface 600).

Process 700 may continue at step 702. At step 702, one or more facialimages of a participant may be captured from each video chat window. Forexample, a facial image may be captured for each of video chat windows601-609 (e.g. facial image 614). In some embodiments, each of video chatwindows 601-609 may correspond to a student accessing an onlineclassroom, which may be displayed on a teacher or host's device (e.g.,host device 108). For example, user interface 600 may be displayed on adisplay screen of a teacher user device.

Process 700 may continue at step 703. After the facial images arecaptured, at step 703, each facial image may be analyzed. Facialrecognition module 222, for example, may analyze each facial image, suchas facial images 611-619, of each video chat window (e.g., video chatwindows 601-609), and may determine a facial expression for eachcorresponding user. In some embodiments, facial recognition module 222may analyze each facial image individually to look for specificcharacteristics of a user's face that correspond to specific emotions.In another embodiment, facial recognition module 222 may compare eachfacial image against a pre-defined facial image associated with acertain emotion.

In one embodiment, the analysis of the facial images may begin with acomparison being performed of each facial image against a plurality ofpredefined facial expressions stored in memory. Predefined facialexpressions may correspond to images of such emotions as: happy,confused, focused, bored, attentive, and/or understood. These predefinedfacial expressions may be stored in a database, such as facialexpression database 224 of device 200. In some embodiments, each storedfacial expression has an associated value. For example, the followingvalues may be associated with some predefined facial expressions storedwithin facial expression database 224: Happy—10; Confused—3; Focused—15;Bored—1; Attentive—12; and Understood—20.

Continuing the example, the analysis of the facial images may continueby matching each facial image to at a predefined facial expression. Tocompare and match facial images to predefined facial expressions, facialrecognition module 222 may look for patterns and/or sequences innumerical data representing each captured facial image. In a capturedfacial image, each part of the image includes pixels. A pixel is aminute area of illumination on a display screen. The system interpretseach image's pixels as a series of numbers. To identify each image, thesystem may identify images by matching it with images having a similarnumerical series. If an image has a similar numerical series as anotherimage above a certain predefined threshold level, then there a matchbetween the two images is determined to be present. A match between twoimages may correspond to more than 75% of the numerical series beingequivalent, however persons of ordinary skill in the art will recognizethat this value is exemplary, and any suitable threshold value may beemployed. So, facial recognition module 222 may interpret a facialimage's as displaying an emotion or feeling based on an emotion orfeeling associated with the matched predefined facial expression. Thecaptured facial image's pixels may, therefore, be compared to the pixelsof various predefined facial expressions.

If a match is determined to be present, a value associated with thatpredefined facial expression is assigned to the facial image matchedthereto. Facial recognition module 222 may continue to compare thefacial image to predefined facial expressions to see if there areadditional matches. For example, a captured facial image from chatwindow 604 may indicate that the corresponding user is displaying ahappy expression and, therefore, the captured image may be assigned avalue of 10 to reflect that the user is happy. Persons having ordinaryskill in the art recognize that the above method of comparing andmatching images is merely exemplary.

Process 700 may continue at step 704. At step 704, the correspondingvalues assigned to each facial image are processed to determine a levelof attentiveness for the participants of the interactive online event.The values may be processed in any number of ways. For example, thevalues may be added together to determine the level of attentiveness forthe interactive online event. Continuing the example where thepredefined facial expression corresponding to the emotion “happy” isassigned a value of 10 (e.g., facial images 611-619 show users that arehappy), the assigned values for each captured facial image would be 10.Processing the assigned values would be accomplished by adding eachvalue (e.g., 10), together. Therefore, the sum of the assigned values inthis example is 90. In some embodiments, the total value for the levelof attentiveness may be used to gauge the overall emotion of theparticipants of the event. For example, if the combined value is greaterthan a certain level, a certain emotion may be attributed to theparticipants. As an illustrative example, a sum greater than 60 maycorrespond to participants being “happy.” Using the example above, thesum of the assigned values is 90, and therefore the determined level ofattentiveness for the interactive online event would be determined to behappy, as the sum is greater than the level for defining the emotionalstate of the group as happy (e.g., 60). While only a sum of the assignedvalues is shown, persons having ordinary skill in the art wouldunderstand that any number of algorithms or methods can be used toprocess the assigned values. The processing may be completed through theuse of processor(s) 202 and facial recognition module 222 of device 200.

In one embodiment, a facial image that was captured may be matched tomore than one predefined facial expression. For example, a user may beboth happy and focused. In this example, there may be a blended valueassigned. If a blended value is used, the blended value may be anaverage between both assigned values. Continuing the example, because apredefined facial expression corresponding to the emotion “happy” has avalue of 10, and the predefined facial expression corresponding to theemotion “focused” has a value of 15, the blended value would be 12.5.Alternatively, the facial image may be counted twice: once as the firstmatched facial expression, and another time as the second matched facialexpression. Using the previous example, the total value would be 25,corresponding to adding the values for happy (e.g., 10) and focused(e.g., 15). Persons of ordinary skill recognize that any suitablecombinatory technique may be employed, and the aforementioned are merelyexemplary.

Process 700 may continue at step 705. At step 705, attentiveness datacorresponding to the level of attentiveness for the participants of theinteractive online event is provided to a host device. Once the assignedvalues are processed and the level of attentiveness is determined, thisdata may be provided to a host device. The attentiveness data may allowthe presenter to refine or modify their presentation so a larger amountof students receive the material being presented, and thus enhancing theoverall experience of each participant. For example, if the interactiveonline event is an online class the host is a teacher, and theparticipants are students, if the level of attentiveness shows thestudents are confused, the teacher may change how he or she is teaching.

In some embodiments, in addition to attentiveness data corresponding tothe level of attentiveness of the participants of the interactive onlineevent, the host may also receive data indicating which predefined facialexpression has been associated with each particular participantaccessing the interactive online event. In some embodiments, theassociated predefined facial expression may be displayed within thehost's displayed user interface to show the video chat window for eachuser with the determined corresponding predefined facial expression. Insome embodiments, this data may be also be stored for later use by ahost. For example, if only one student is not understanding thematerial, the host may continue the interactive online event toaccommodate the majority of students. Once the interactive online eventis over, the host may look at the data and revisit the material with theparticipant that did not understand the material.

Process 700 may be used in situations where the online event isprerecorded. In this case, each participant's level of attentiveness maybe provided to the host during or after the participant has completedthe online event.

In some embodiments, process 700 may be a dynamic process that updatesthe level of attentiveness and provides it to the host in real time.This may allow the host to continually receive feedback regarding theeffectiveness of their presented materials, and modify theirpresentation techniques accordingly through the presentation's duration.Additionally, the attentiveness data may be stored so the host of theinteractive online event can review how participants reacted throughoutthe online event. Furthermore, process 700 may be repeated a pluralityof times throughout the presentation. In this scenario, captured facialimages may be compared with one another to measure the displacement ofcertain facial characteristics. In this particular embodiment, a facialexpression, and therefore a level of attentiveness, may be determinedfrom the measured displacement of facial characteristics. As anillustrative example, at a first time, an individual's face image maydisplay a smile. At a second time, the individual's face may display aslightly different configuration of that individual's mouth. Thedifference between the position of the mouth corresponding to the smileand the new configuration may be analyzed to determine a displacement ofthe individual's mouth. Certain emotional states may, therefore, beassociated with different displacements for different features. In theaforementioned example, the determined displacement may indicate thatthe participant has gone from a happy state to a confused state.

FIG. 8 is an illustrative flowchart of an exemplary process inaccordance with various embodiments. Process 800 may begin at step 801.At step 801, a video may be received from a participant accessing aninteractive online event. For example, the interactive online event maybe an online class, the participant may be a student, and the host maybe a teacher. In some embodiments, step 801 of process 800 may besubstantially similar to step 701 of process 700, and the previousdescription may apply.

Process 800 may continue at step 802. At step 802, a facial image fromthe received video of a participant may be captured. For example, facialimage 614 may be captured from video chat window 604. In someembodiments, a video chat window may correspond to a student accessingan online classroom, which may be displayed on a teacher or host'sdevice. For example, user interface 600 may be displayed on a displayscreen of a teacher or host's device. Step 802 of process 800, in someembodiments, may be substantially similar to step 702 of process 700,and the previous description may also apply.

Process 800 may continue at step 803. After the facial image has beencaptured, at step 803 the facial image may be analyzed. A facialrecognition module, such as facial recognition module 222 of device 200,may analyze the captured facial image (e.g., facial image 614 from videochat window 604). In some embodiments, facial recognition module 222 mayanalyze the captured facial image to look for specific characteristicsof a user's face that correspond to specific emotions, or facialrecognition module 222 may compare the facial image against apre-defined facial image associated with a certain emotion (e.g.,predefined facial expression).

In one embodiment, the analysis of the facial image starts with acomparison being performed for the facial image against a variety ofpredefined facial expressions stored in memory. Predefined facialexpressions may correspond to images of such emotions as: happy,confused, focused, bored, attentive, and/or understood. These predefinedfacial expressions may be stored in a database, such as facialexpression database 224 of device 200. In some embodiments, each storedfacial expression has an associated value. For example, the followingvalues may be associated with some particular facial expression havingpredefined facial expressions stored within facial expression database224: Happy—10; Confused—3; Focused—15; Bored—1; Attentive—12; andUnderstood—20.

Continuing the example, the analysis of the facial image continues bydetermining at least one predefined facial expression the facial image.When a match is determined to occur between a facial expression and thefacial image, the value associated with that matched facial expressionis assigned to the facial image. For example, the facial expressioncorresponding to the emotion “Happy” may have a value—10, and thereforethe captured image 614 would be assigned the value of 10 to reflect thatthe user is happy.

Process 800 may continue at step 804. At step 804, the correspondingvalue assigned to the facial image is processed to determine a level ofattentiveness for the participant of the interactive online event. Thevalue may be processed in any number of ways. For example, the valuesmay be compared to the assigned values to each facial expression.Continuing the above example, because captured image 614 was assigned avalue of 10, the level of attentiveness for the participant would beHappy. In some embodiments, step 804 of process 800 may be substantiallysimilar to step 704 of process 700, and the previous description mayapply.

Process 800 may continue at step 805. At step 805, the level ofattentiveness for the participant of the interactive online event may beprovided to a host device. Continuing the above example, the host wouldbe provided with a determined level of attentiveness of “Happy.” Theattentiveness data may allow the presenter to refine or modify theirpresentation so a larger amount of students comprehend the materialbeing presented, and thus enhancing the overall experience of eachparticipant. For example, if the interactive online event is an onlineclass, the host is a teacher, the participants are students, and thelevel of attentiveness indicates that the students are confused, theteacher may decide to change how he or she is teaching based on theattentiveness data. In some embodiments, step 805 of process 800 may besubstantially similar to step 705 of process 700, and the previousdescription may apply.

In some embodiments, in addition to the level of attentiveness of theinteractive online event, the host may also receive data showing eachmatched predefined facial expression. Furthermore, the data may alsoshow the particular participant associated with that the matched facialexpression.

Process 800 may be used in situations where the online event isprerecorded. In this case, the participant's level of attentiveness maybe provided during or after the participant has completed the onlineevent.

In some embodiments, process 800 may be a dynamic process that updatesthe level of attentiveness and provides it to the host in real time.This may allow the host to continually receive feedback regarding theeffectiveness of their presented materials, and modify theirpresentation techniques accordingly through the presentation's duration.Additionally, the attentiveness data may be stored so the host of theinteractive online event can review how participants reacted throughoutthe online event. Furthermore, process 800 may be repeated a pluralityof times throughout the presentation. In this scenario, captured facialimages may be compared with one another to measure the displacement ofcertain facial characteristics. In this particular embodiment, a facialexpression, and therefore a level of attentiveness, may be determinedfrom the measured displacement of facial characteristics. As anillustrative example, at a first time, an individual's face image maydisplay a smile. At a second time, the individual's face may display aslightly different configuration of that individual's mouth. Thedifference between the position of the mouth corresponding to the smileand the new configuration may be analyzed to determine a displacement ofthe individual's mouth. Certain emotional states may, therefore, beassociated with different displacements for different features. In theaforementioned example, the determined displacement may indicate thatthe participant has gone from a happy state to a confused state.

In some embodiments, providing assistance to individuals expressingconfusion or other negative emotions may be enhanced. For example,enhanced analytics of which participants within the interactive onlineevent are experiencing difficulty may be ascertained via the facialrecognition module. This information may be used in real-time to helpthose students to better understand the material being presented.Furthermore, the information regarding attentiveness may be usedasynchronously with the presentation of the material to allow a teacheror presenter to contact individuals who expressed confusion during thepresentation, and reach out and help them.

In some embodiments, the facial recognition module may run during a livepresentation or during a recorded teaching. For example, the facialexpressions of students within an online class may be analyzed andquantified based on the presented materials. Therefore, in at least oneembodiment, the presenter, such as a teacher, may present materials tothe students live or from a pre-recorded video.

In some embodiments, the facial recognition module may continually runthroughout the span of the presentation to determine a cycle ofexpressions or emotions displayed by an individual. For example, if astudent is expressing confusion for a prolonged period of time, thatdata may be provided to the teacher so that the teacher may attempt toaid the student. This may be extremely helpful because some students mayonly suffer from confusion for a short period of time, while others maysuffer from confusion for longer periods of time. The students sufferingconfusion for short periods of time may be able to quickly resolve theirconfusion, whereas students suffering confusion for longer periods oftime may require additional help or guidance.

Furthermore, in some embodiments, educational institutions, such asuniversities or high schools, may use the analyzed data to determineteacher or presenter effectiveness. For example, if a teachercontinually receives confused facial expressions, that teacher may notbe performing adequately, and therefore the institution the teacherworks for may assess the teacher's effectiveness accordingly. This maybe extremely useful for institutions that want quantitative datacorresponding to teacher effectiveness that are not measurable solely byusing a teachers grading system or grading history, as a lenient teachermay garner high marks based on their grading history, but may generallybe ineffective in teaching the required materials.

The various embodiments described herein may be implemented using avariety of means including, but not limited to, software, hardware,and/or a combination of software and hardware. The embodiments may alsobe embodied as computer readable code on a computer readable medium. Thecomputer readable medium may be any data storage device that is capableof storing data that can be read by a computer system. Various types ofcomputer readable media include, but are not limited to, read-onlymemory, random-access memory, CD-ROMs, DVDs, magnetic tape, or opticaldata storage devices, or any other type of medium, or any combinationthereof. The computer readable medium may be distributed overnetwork-coupled computer systems. Furthermore, the above describedembodiments are presented for the purposes of illustration are not to beconstrued as limitations.

What is claimed is:
 1. A method for monitoring participants' level ofattentiveness within an interactive online event, the method comprising:receiving, from each of a plurality of participants accessing aninteractive online event, at least one video; capturing, from eachreceived video, at least one facial image; analyzing each capturedfacial image, wherein analyzing comprises: comparing each capturedfacial image to a plurality of predefined facial expressions; andmatching each captured facial image to at least one predefined facialexpression, wherein each match assigns a value to the captured facialimage; determining a level of attentiveness for the interactive onlineevent by processing each of the assigned values together; and providingattentiveness data representing the determined level of attentiveness toa host device accessing the interactive online event.
 2. The method ofclaim 1, further comprising: providing, to the host device, access toeach matched facial expression.
 3. The method of claim 1, furthercomprising: generating a plurality of indicators, each indicatorrepresenting the value assigned to each captured image; and providing,to the host device, access to the generated plurality of indicators. 4.The method of claim 1, further comprising: changing a color of a userinterface displayed on the host device in response to the attentivenessdata.
 5. The method of claim 1, further comprising: providing a messagerepresenting the attentiveness data to the host device.
 6. The method ofclaim 5, wherein the message comprises at least one of: happy, confused,focused, bored, attentive, and understood.
 7. The method of claim 1,wherein: the interactive online event is an online class; the pluralityof participants are students accessing the online class; and the hostdevice is a terminal accessible to a teacher of the online class.
 8. Themethod of claim 1, wherein the plurality of predefined facialexpressions comprises an emotion of at least one of: happy, confused,focused, bored, attentive, and understood.
 9. A method for monitoring aparticipant's level of attentiveness within an interactive online event,the method comprising: A) receiving video from a participant accessingan interactive online event; B) capturing a facial image from thereceived video; C) analyzing the captured facial image, whereinanalyzing comprises: comparing the captured facial image to a pluralityof predefined facial expressions; and matching the captured facial imageto at least one predefined facial expression wherein each match assignsa value to the captured facial image; D) determining a level ofattentiveness of the interactive online event by processing eachassigned value; and E) providing the determined level of attentivenessto a host device accessing the interactive online event.
 10. The methodof claim 9, wherein steps B through E are repeated a plurality of timesduring the interactive online event.
 11. The method of claim 9, furthercomprising: providing, to the host device, access to each matched typeof facial expression.
 12. The method of claim 9, further comprising:generating an indicator, the indicator representing the value assignedto the captured image; and providing, to the host device, access to thegenerated indicator.
 13. The method of claim 9, further comprising:changing the color of a user interface of the host device in response tothe level of attentiveness.
 14. The method of claim 9, wherein the datafurther comprises a message stating the level of attentiveness.
 15. Themethod of claim 14, wherein the message comprises at least one of:happy, confused, focused, bored, attentive, and understood.
 16. Themethod of claim 9, wherein: the interactive online event is an onlineclass; the participant is a student accessing the online class; and thehost device is a terminal accessible to a teacher of the online class.17. The method of claim 9, wherein the plurality of types of facialexpressions comprises at least one of: happy, confused, focused, bored,attentive, and understood.
 18. A system for monitoring participants'level of attentiveness within an interactive online event, the systemcomprising: a plurality of user devices accessing an interactive onlineevent, wherein each user device corresponds to a participant of theinteractive online event; a host device accessing the interactive onlineevent, wherein the host device corresponds to a host of the interactiveonline event; a server, the server operable to: receive, from each ofthe plurality of user devices, at least one video; capture, from eachreceived video, at least one facial image; analyze each captured facialimage, wherein analyze comprises: compare each captured facial image toa plurality of predefined facial expressions; and match each capturedfacial image to at least one predefined facial expression, wherein eachmatch assigns a value to the captured facial image; determine a level ofattentiveness for the interactive online event by processing each of theassigned values together; and transmit the determined level ofattentiveness to the host device.
 19. The system of claim 18, the serverfurther operable to: change the color of a user interface of the hostdevice in response to the level of attentiveness.
 20. The method ofclaim 18, wherein: the interactive online event is an online class; theplurality of participants are students accessing the online class; andthe host is a teacher of the online class.